{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Movie Review Explanations\n",
    "\n",
    "\n",
    "We will use an SKLearn classifier built on [movie sentiment data](http://www.cs.cornell.edu/people/pabo/movie%2Dreview%2Ddata/) which predicts positive or negative sentiment for review text. \n",
    "\n",
    "The Kfserving resource provides:\n",
    "   * A pretrained sklearn model stored on a Google bucket\n",
    "   * A Text [Seldon Alibi](https://github.com/SeldonIO/alibi) Explainer. See the [Alibi Docs](https://docs.seldon.io/projects/alibi/en/stable/) for further details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pygmentize moviesentiment.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl apply -f moviesentiment.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "CLUSTER_IPS = !(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')\n",
    "CLUSTER_IP = CLUSTER_IPS[0]\n",
    "print(CLUSTER_IP)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SERVICE_HOSTNAMES = !(kubectl get inferenceservice moviesentiment -o jsonpath='{.status.url}' | cut -d \"/\" -f 3)\n",
    "SERVICE_HOSTNAME = SERVICE_HOSTNAMES[0]\n",
    "print(SERVICE_HOSTNAME)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "sys.path.append(\"../\")\n",
    "from alibi_helper import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from alibi.datasets import fetch_movie_sentiment\n",
    "\n",
    "movies = fetch_movie_sentiment()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "idxNeg = 37\n",
    "idxPos = 5227\n",
    "for idx in [idxNeg, idxPos]:\n",
    "    print(movies.data[idx])\n",
    "    show_prediction(\n",
    "        predict(\n",
    "            movies.data[idx : idx + 1],\n",
    "            \"moviesentiment\",\n",
    "            movies,\n",
    "            SERVICE_HOSTNAME,\n",
    "            CLUSTER_IP,\n",
    "        )\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get Explanation for Negative Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "exp = explain(\n",
    "    movies.data[idxNeg : idxNeg + 1], \"moviesentiment\", SERVICE_HOSTNAME, CLUSTER_IP\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "show_anchors(exp[\"data\"][\"anchor\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show precision. How likely predictions using the Anchor features would produce the same result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "show_bar([exp[\"data\"][\"precision\"]], [\"\"], \"Precision\")\n",
    "show_bar([exp[\"data\"][\"coverage\"]], [\"\"], \"Coverage\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "show_feature_coverage(exp[\"data\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "show_examples(exp[\"data\"], 0, movies)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "show_examples(exp[\"data\"], 0, movies, False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get Explanation for Positive Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "exp = explain(\n",
    "    movies.data[idxPos : idxPos + 1], \"moviesentiment\", SERVICE_HOSTNAME, CLUSTER_IP\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "show_anchors(exp[\"data\"][\"anchor\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show precision. How likely predictions using the Anchor features would produce the same result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "show_bar([exp[\"data\"][\"precision\"]], [\"\"], \"Precision\")\n",
    "show_bar([exp[\"data\"][\"coverage\"]], [\"\"], \"Coverage\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "show_feature_coverage(exp[\"data\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "show_examples(exp[\"data\"], 0, movies)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "show_examples(exp[\"data\"], 0, movies, False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Teardown"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl delete -f moviesentiment.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
